The exponential growth of the Internet has made it a ubiquitous delivery medium for a variety of applications. These applications have in turn brought with them an increasing demand for bandwidth. As a result, service providers race to build larger and faster data centers with versatile capabilities. Meanwhile, advances in virtualization technologies have made it possible to implement a large number of virtual machines (VMs) in a data center. These virtual machines can essentially operate as physical hosts and perform a variety of functions such as Web or database servers. Because virtual machines are implemented in software, they can freely migrate to various locations. This capability allows service providers to partition and isolate physical resources (e.g., computing power and network capacity) according to customer needs, and to allocate such resources dynamically.
While virtualization brings unprecedented flexibility to service providers, the conventional layer-2 network architecture, however, tends to be rigid and cannot readily accommodate the dynamic nature of virtual machines. For example, in conventional data center architecture, hosts are often inter-connected by one or more layer-2 (e.g., Ethernet) switches to form a layer-2 broadcast domain. The physical reach of a layer-2 broadcast domain is limited by the transmission medium. As a result, different data centers are typically associated with different layer-2 broadcast domains, and multiple layer-2 broadcast domains could exist within a single data center. For a VM in one data center to communicate with a VM or a storage device in another data center, such communication would need to be carried over layer-3 networks. That is, the packets between the source and destination have to be processed and forwarded by layer-3 devices (e.g., IP routers), since the source and destination belong to different layer-2 broadcast domains. While this architecture has benefits, flat layer-2 processing has its advantages.
One technique to solve the problems described above is to implement a virtual extensible local area network (VXLAN). VXLAN is a standard network virtualization technology managed by the Internet Engineering Task Force (IETF), and works by creating a logical layer-2 network that is overlaid above a layer-3 IP network. Ethernet packets generated by VMs are encapsulated in an IP header before they are transported to a remote location where the IP header is removed and the original Ethernet packet is delivered to the destination. The IP encapsulation mechanism allows a logical layer-2 broadcast domain to be extended to an arbitrary number of remote locations, and allows different data centers or different sections of the same data center (and hence the VMs and devices therein) to be in the same layer-2 broadcast domain. The VXLAN function typically resides within a host's hypervisor, and works in conjunction with the hypervisor's virtual switch. More details of VXLAN can be found in IETF draft “VXLAN: A Framework for Overlaying Virtualized Layer 2 Networks over Layer 3 Networks,” which is incorporated by reference here.
Existing VXLAN implementations, however, cannot readily take advantage of some of the hardware-based off-loading features available in the physical network interface cards (PNICs). For example, certain types of PNICs allow allocation of separate receive queues for packets destined to different VMs based on their MAC address and/or virtual local area network (VLAN) tags, which can facilitate multi-core processing of the received packets and improve the throughput while reducing processing overhead on the CPUs. However, due to the nature of VXLAN encapsulation, the VXLAN packets received by a PNIC cannot readily benefit from such queuing.